Design of a Virtual Multi-Interaction Operation System for Hand–Eye Coordination of Grape Harvesting Robots

In harvesting operations, simulation verification of hand–eye coordination in a virtual canopy is critical for robot research. More realistic scenarios, vision-based driving motion, and cross-platform interaction information are needed to achieve such simulations, which very challenging. Current simulations more focused on path planning operations consistency far from satisfying the requirements. To this end, new approach visual servo multi-interaction real scenarios proposed. study, dual-arm grape laboratory used as an example. overcome these challenges, multi-software federation first proposed establish their communication cross-software sending image information, coordinate control commands. Then, fruit recognition positioning algorithm, forward inverse kinematic model embedded OpenCV MATLAB, respectively, drive run V-REP, thus realizing trellis vineyard. Finally, verified, results show that average running time string-picking system 6.5 s, success rate accurate picking point grasping reached 83.3%. A complex closed loop “scene-image recognition-grasping” formed by data processing transmission various information. It can effectively realize continuous under environment.